Skip to content

feat: Add Zarr v2 metadata access and basic auth support for OIDC-protected assets#2112

Draft
cauriol wants to merge 16 commits into
developfrom
auth_destinee
Draft

feat: Add Zarr v2 metadata access and basic auth support for OIDC-protected assets#2112
cauriol wants to merge 16 commits into
developfrom
auth_destinee

Conversation

@cauriol

@cauriol cauriol commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator

This PR adds support for accessing Zarr v2 assets from EOProduct and extends OIDC authentication handling to support Basic authorization for protected asset access.

Changes:
-add helper methods in EOProduct to:
- extract auth headers from auth objects
- request protected assets with forwarded authentication headers
- list files from Zarr v2 metadata using .zmetadata

  • extend CodeAuthorizedAuth usage to support Basic authorization headers for protected asset access
  • protect OIDC authentication token retrieval with a lock to avoid concurrent token refresh issues

Notes
this PR currently targets Zarr v2 only through .zmetadata
Zarr v3 support can be added later if needed

@cauriol cauriol self-assigned this Mar 25, 2026
@cauriol cauriol changed the title Refactor: CodeAuthorizedAuth headers handling for header/basic auth refactor: CodeAuthorizedAuth headers handling for header/basic auth Mar 25, 2026
@github-actions

github-actions Bot commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Test Results

  1 files   -     3    1 suites   - 3   33s ⏱️ - 3m 14s
853 tests +   10  849 ✅ +    6  0 💤 ±0  4 ❌ +4 
878 runs   - 2 544  874 ✅  - 2 546  0 💤  - 2  4 ❌ +4 

For more details on these failures, see this check.

Results for commit d4f4729. ± Comparison against base commit 8c36c36.

♻️ This comment has been updated with latest results.

@eodag-bot

eodag-bot commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator

badge

Code Coverage (Ubuntu)

Details
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         251      11  95.62%   104-115, 377, 644
config.py                                      306      24  92.16%   70-72, 75, 78, 81, 85, 89, 93-95, 587-589, 711-713, 732, 740, 770-775, 777
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    774      57  92.64%   263, 549, 597, 640, 680, 700, 741-746, 771, 855-874, 888, 894, 1047, 1052, 1151, 1190-1191, 1287-1288, 1314, 1345-1346, 1372, 1385, 1446-1447, 1478-1479, 1553-1558, 1570-1573, 1685, 1910, 2067, 2178, 2266-2267
api/provider.py                                382      40  89.53%   186, 190-191, 316-337, 405, 521, 525-526, 531-534, 545, 621-631, 821-822, 870, 877, 891, 914-917, 951-958, 962-963
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 413, 468-471
api/product/__init__.py                          7       0  100.00%
api/product/_assets.py                          54       4  92.59%   97, 191, 202-206
api/product/_product.py                        344      24  93.02%   208, 364, 390-393, 430, 441-442, 551, 580, 587, 626, 797, 840-843, 852-855, 909, 975, 987, 1054
api/product/metadata_mapping.py                806      60  92.56%   125-127, 221-226, 250, 308-309, 355-356, 397, 418, 470-471, 508, 529-532, 555, 567-568, 609, 632, 657-660, 662-667, 732-737, 746, 752, 768, 776, 1010, 1163, 1185, 1194-1198, 1215-1220, 1353, 1376, 1385, 1407, 1412, 1464, 1536, 1557, 1583, 1597, 1622, 1668, 1737, 1812
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 25       2  92.00%   48, 55
plugins/manager.py                             173      16  90.75%   102-107, 179, 201, 219-220, 232, 271-272, 372-375, 387-388
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           95       8  91.58%   163-165, 213-214, 240-242
plugins/apis/usgs.py                           182      25  86.26%   157, 263, 297, 339-341, 346, 374-375, 380, 410-417, 428-433, 455-461
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  22       4  81.82%   45, 58, 81, 95
plugins/authentication/eoiam.py                 99       2  97.98%   171, 194
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   59       0  100.00%
plugins/crunch/filter_latest_intersect.py       54       6  88.89%   92-93, 100-101, 103-107
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       0  100.00%
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        402      76  81.09%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-834, 846-851, 889, 915, 960-962, 1014
plugins/download/base.py                       288      24  91.67%   135, 165, 371-372, 416, 450, 527-531, 561, 596-597, 622-631, 690, 711, 733, 741
plugins/download/http.py                       575      74  87.13%   235, 277-280, 342-345, 348, 355-360, 391-393, 410, 425, 486, 521, 535, 549, 559-563, 579-584, 595, 614, 651-654, 675, 685, 692, 748, 857, 889, 921-930, 966, 1011-1016, 1025, 1040-1042, 1046, 1049, 1064-1065, 1075, 1150, 1202, 1244-1245, 1257, 1267, 1323-1324, 1354, 1392, 1472-1473
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         197      18  90.86%   109, 113, 137-143, 218-221, 296, 317, 443, 493, 526-529, 538
plugins/search/build_search_result.py          519      91  82.47%   133-134, 170, 174, 194, 253, 423-434, 444-446, 532-538, 584, 608, 610, 677, 685-689, 710, 720, 746, 791, 816, 844, 862-877, 927, 952, 955, 959, 968, 974, 1012-1033, 1074, 1101-1102, 1111-1120, 1184, 1199, 1205, 1224-1233, 1354-1355, 1399, 1408-1410, 1467, 1515-1525
plugins/search/cop_ghsl.py                     407      85  79.12%   73-74, 109-110, 174, 176, 218, 249-252, 297, 313, 316, 350, 354-359, 374, 396, 417, 451, 467, 502, 517, 531-532, 546-548, 555, 559-561, 564-567, 585-645, 656-671
plugins/search/cop_marine.py                   273      59  78.39%   57, 65-67, 77-78, 83, 88-89, 105, 107, 110, 176-177, 220, 238, 244, 248, 252, 263, 274-275, 283, 315-318, 324, 334, 347, 351, 355, 359, 363-367, 373-376, 379-396, 413-416, 469-473, 478, 490, 504-505
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 247-288
plugins/search/geodes.py                        45       0  100.00%
plugins/search/qssearch.py                     865      98  88.67%   382, 427-428, 545-546, 569-570, 582-586, 817-823, 881, 947, 977, 984, 1055, 1076, 1079-1080, 1098, 1107-1108, 1135, 1207, 1216, 1221-1238, 1247, 1262, 1271, 1284, 1306, 1402, 1425, 1440, 1513-1514, 1516-1518, 1520-1521, 1608, 1615, 1723-1727, 1798, 1801, 1805-1806, 1827-1830, 1842, 1864-1876, 1884, 1919-1921, 1944-1950, 1957, 2011, 2034, 2039-2040, 2055, 2061, 2075, 2176, 2180, 2191, 2215, 2228, 2236-2246
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      18  78.57%   99-127, 166-169, 182, 224
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 285, 295-311, 316, 318, 340, 345, 353, 363
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            112       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/stac_extensions.py                       148       4  97.30%   432-438, 619
types/stac_metadata.py                         154      22  85.71%   95, 117, 150, 183-185, 199-213, 222-229, 256-259
utils/__init__.py                              572      39  93.18%   60, 198, 229-230, 239-265, 268, 283, 363-367, 442-446, 526, 566-567, 596, 974-977, 1028, 1047-1048, 1077, 1095-1096, 1208, 1296, 1460, 1698
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 182       4  97.80%   172, 516, 523-524
utils/deserialize.py                            73      10  86.30%   53, 55, 103, 108-109, 145, 159, 181-182, 188
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                31       1  96.77%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    240      12  95.00%   200-203, 245, 263, 489, 537-538, 585, 660, 686
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         82       7  91.46%   24-26, 63, 124, 133, 145
TOTAL                                        11096    1341  87.91%

Diff against develop

Filename                                    Stmts    Miss  Cover
----------------------------------------  -------  ------  -------
api/product/_product.py                       +37       0  +0.84%
plugins/authentication/openid_connect.py       +8      +2  -0.55%
TOTAL                                         +45      +2  +0.03%

Results for commit: 9faf04a

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@eodag-bot

eodag-bot commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator

badge

Code Coverage (Windows)

Details
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         251      11  95.62%   104-115, 377, 644
config.py                                      306      24  92.16%   70-72, 75, 78, 81, 85, 89, 93-95, 587-589, 711-713, 732, 740, 770-775, 777
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    774      57  92.64%   263, 549, 597, 640, 680, 700, 741-746, 771, 855-874, 888, 894, 1047, 1052, 1151, 1190-1191, 1287-1288, 1314, 1345-1346, 1372, 1385, 1446-1447, 1478-1479, 1553-1558, 1570-1573, 1685, 1910, 2067, 2178, 2266-2267
api/provider.py                                382      40  89.53%   186, 190-191, 316-337, 405, 521, 525-526, 531-534, 545, 621-631, 821-822, 870, 877, 891, 914-917, 951-958, 962-963
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 413, 468-471
api/product/__init__.py                          7       0  100.00%
api/product/_assets.py                          54       4  92.59%   97, 191, 202-206
api/product/_product.py                        344      24  93.02%   208, 364, 390-393, 430, 441-442, 551, 580, 587, 626, 797, 840-843, 852-855, 909, 975, 987, 1054
api/product/metadata_mapping.py                806      60  92.56%   125-127, 221-226, 250, 308-309, 355-356, 397, 418, 470-471, 508, 529-532, 555, 567-568, 609, 632, 657-660, 662-667, 732-737, 746, 752, 768, 776, 1010, 1163, 1185, 1194-1198, 1215-1220, 1353, 1376, 1385, 1407, 1412, 1464, 1536, 1557, 1583, 1597, 1622, 1668, 1737, 1812
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 25       2  92.00%   48, 55
plugins/manager.py                             173      16  90.75%   102-107, 179, 201, 219-220, 232, 271-272, 372-375, 387-388
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           95       8  91.58%   163-165, 213-214, 240-242
plugins/apis/usgs.py                           182      25  86.26%   157, 263, 297, 339-341, 346, 374-375, 380, 410-417, 428-433, 455-461
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  22       4  81.82%   45, 58, 81, 95
plugins/authentication/eoiam.py                 99       2  97.98%   171, 194
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   59       0  100.00%
plugins/crunch/filter_latest_intersect.py       54       6  88.89%   92-93, 100-101, 103-107
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       0  100.00%
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        402      76  81.09%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-834, 846-851, 889, 915, 960-962, 1014
plugins/download/base.py                       288      26  90.97%   135, 165, 232-234, 371-372, 416, 450, 527-531, 561, 596-597, 622-631, 690, 711, 733, 741
plugins/download/http.py                       575      74  87.13%   235, 277-280, 342-345, 348, 355-360, 391-393, 410, 425, 486, 521, 535, 549, 559-563, 579-584, 595, 614, 651-654, 675, 685, 692, 748, 857, 889, 921-930, 966, 1011-1016, 1025, 1040-1042, 1046, 1049, 1064-1065, 1075, 1150, 1202, 1244-1245, 1257, 1267, 1323-1324, 1354, 1392, 1472-1473
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         197      18  90.86%   109, 113, 137-143, 218-221, 296, 317, 443, 493, 526-529, 538
plugins/search/build_search_result.py          519      91  82.47%   133-134, 170, 174, 194, 253, 423-434, 444-446, 532-538, 584, 608, 610, 677, 685-689, 710, 720, 746, 791, 816, 844, 862-877, 927, 952, 955, 959, 968, 974, 1012-1033, 1074, 1101-1102, 1111-1120, 1184, 1199, 1205, 1224-1233, 1354-1355, 1399, 1408-1410, 1467, 1515-1525
plugins/search/cop_ghsl.py                     407      85  79.12%   73-74, 109-110, 174, 176, 218, 249-252, 297, 313, 316, 350, 354-359, 374, 396, 417, 451, 467, 502, 517, 531-532, 546-548, 555, 559-561, 564-567, 585-645, 656-671
plugins/search/cop_marine.py                   273      59  78.39%   57, 65-67, 77-78, 83, 88-89, 105, 107, 110, 176-177, 220, 238, 244, 248, 252, 263, 274-275, 283, 315-318, 324, 334, 347, 351, 355, 359, 363-367, 373-376, 379-396, 413-416, 469-473, 478, 490, 504-505
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 247-288
plugins/search/geodes.py                        45       0  100.00%
plugins/search/qssearch.py                     865      98  88.67%   382, 427-428, 545-546, 569-570, 582-586, 817-823, 881, 947, 977, 984, 1055, 1076, 1079-1080, 1098, 1107-1108, 1135, 1207, 1216, 1221-1238, 1247, 1262, 1271, 1284, 1306, 1402, 1425, 1440, 1513-1514, 1516-1518, 1520-1521, 1608, 1615, 1723-1727, 1798, 1801, 1805-1806, 1827-1830, 1842, 1864-1876, 1884, 1919-1921, 1944-1950, 1957, 2011, 2034, 2039-2040, 2055, 2061, 2075, 2176, 2180, 2191, 2215, 2228, 2236-2246
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      18  78.57%   99-127, 166-169, 182, 224
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 285, 295-311, 316, 318, 340, 345, 353, 363
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            112       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/stac_extensions.py                       148       4  97.30%   432-438, 619
types/stac_metadata.py                         154      22  85.71%   95, 117, 150, 183-185, 199-213, 222-229, 256-259
utils/__init__.py                              572      39  93.18%   60, 198, 229-230, 239-265, 268, 283, 363-367, 442-446, 526, 566-567, 596, 974-977, 1028, 1047-1048, 1077, 1095-1096, 1208, 1296, 1460, 1698
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 182       4  97.80%   172, 516, 523-524
utils/deserialize.py                            73      10  86.30%   53, 55, 103, 108-109, 145, 159, 181-182, 188
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                31       1  96.77%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    240      12  95.00%   200-203, 245, 263, 489, 537-538, 585, 660, 686
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         82       7  91.46%   24-26, 63, 124, 133, 145
TOTAL                                        11096    1343  87.90%

Diff against develop

Filename                                    Stmts    Miss  Cover
----------------------------------------  -------  ------  -------
api/product/_product.py                       +37       0  +0.84%
plugins/authentication/openid_connect.py       +8      +2  -0.55%
TOTAL                                         +45      +2  +0.03%

Results for commit: 9faf04a

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@cauriol
cauriol marked this pull request as ready for review March 26, 2026 09:37
@cauriol
cauriol requested a review from jlahovnik March 26, 2026 09:37
@cauriol
cauriol marked this pull request as draft April 1, 2026 09:27
@cauriol
cauriol removed the request for review from jlahovnik April 1, 2026 09:27
@cauriol cauriol changed the title refactor: CodeAuthorizedAuth headers handling for header/basic auth feat: Add Zarr v2 metadata access and basic auth support for OIDC-protected assets Apr 1, 2026
@alambare

alambare commented Apr 1, 2026

Copy link
Copy Markdown
Collaborator

@cauriol, instead of introducing a new method get_auth_headers in plugin configs, we should leverage the product._get_storage_options from EODAG cube: https://github.com/CS-SI/eodag-cube/blob/develop/eodag_cube/api/product/_product.py#L121.
This method should probably be refactored to not activate order and become a public interface.

@sbrunato can we get your opinion here ?

@sbrunato

sbrunato commented Apr 2, 2026

Copy link
Copy Markdown
Collaborator

@cauriol, instead of introducing a new method get_auth_headers in plugin configs, we should leverage the product._get_storage_options from EODAG cube: https://github.com/CS-SI/eodag-cube/blob/develop/eodag_cube/api/product/_product.py#L121. This method should probably be refactored to not activate order and become a public interface.

@sbrunato can we get your opinion here ?

@alambare I agree. EOProduct._get_storage_options can be moved from eodag-cube to eodag and publicly exposed

@cauriol
cauriol force-pushed the auth_destinee branch 2 times, most recently from 141c247 to ca62273 Compare April 2, 2026 12:57
@cauriol
cauriol marked this pull request as ready for review April 2, 2026 13:36
@cauriol
cauriol requested a review from alambare April 2, 2026 13:36
Comment on lines +589 to +595
def request_asset(
self,
url: str,
) -> requests.Response:
"""Perform a GET request to the given URL using product's authentication headers."""
headers = self.get_storage_options().get("headers", {})
return requests.get(url, headers=headers, stream=True)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should use the stream download method from EODAG download plugins instead of creating a new method.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

method to remove

Comment thread eodag/api/product/_product.py Outdated
Comment thread eodag/api/product/_product.py Outdated
Comment thread eodag/api/product/_product.py Outdated
Comment thread eodag/api/product/_product.py Outdated
Comment thread pyproject.toml Outdated
Comment thread eodag/api/product/_product.py Outdated
@cauriol
cauriol marked this pull request as draft April 10, 2026 12:10
@cauriol
cauriol marked this pull request as ready for review April 13, 2026 07:46
@cauriol
cauriol requested a review from alambare April 13, 2026 08:03

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_storage_options implementation should be done in a separate PR

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has been extracted to #2276

Comment on lines +589 to +595
def request_asset(
self,
url: str,
) -> requests.Response:
"""Perform a GET request to the given URL using product's authentication headers."""
headers = self.get_storage_options().get("headers", {})
return requests.get(url, headers=headers, stream=True)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

method to remove

Comment thread eodag/resources/providers.yml Outdated
authentication_uri_source: login-form
---
!provider # MARK: desp_cache
name: desp_cache

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provider name and description need to be updated. "Cache" does not seem appropriate in name and description.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After further consideration I think we cannot add this provider in EODAG because it the data is in internal FDP which is not publicly exposed. But we can add the collections to the dedl provider and we should keep the changes for the auth plugin.

Comment thread eodag/resources/providers.yml Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provider and its registration are not documented

@sbrunato
sbrunato marked this pull request as draft July 10, 2026 10:27
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

badge

Code Coverage (Ubuntu)

Details
Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      27  91.96%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 746-748, 772-776, 795, 803, 833-838, 840
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    776      58  92.53%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1150, 1189-1190, 1286-1287, 1313, 1344-1345, 1371, 1384, 1445-1446, 1477-1478, 1552-1557, 1569-1572, 1684, 1909, 2066, 2070, 2179, 2267-2268
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 846-847, 895, 902, 916, 939-942, 976-983, 987-988
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 413, 468-471
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          58       5  91.38%   101, 195-196, 208-212
api/product/_product.py                        352      24  93.18%   226, 382, 408-411, 448, 467-468, 577, 606, 613, 653, 824, 867-870, 879-882, 936, 1002, 1014, 1081
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1972, 2038, 2082
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 413-416, 428-429
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   58, 71, 94, 108
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              57       3  94.74%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        403      77  80.89%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       297      24  91.92%   148, 178, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       578      74  87.20%   242, 284-287, 349-352, 355, 362-367, 398-400, 417, 432, 493, 528, 542, 556, 566-570, 586-591, 602, 621, 657-660, 681, 691, 698, 754, 862, 894, 926-935, 971, 1016-1021, 1030, 1045-1047, 1051, 1054, 1069-1070, 1080, 1155, 1207, 1249-1250, 1262, 1272, 1328-1329, 1359, 1397, 1477-1478
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         207      19  90.82%   118, 122, 146-152, 161, 227-230, 290, 322, 343, 469, 554-557, 566
plugins/search/build_search_result.py          527      78  85.20%   138-139, 175, 179, 199, 258, 427-438, 517-518, 567, 579-581, 605, 607, 672, 680-684, 704, 714, 740, 789, 814, 876-881, 942, 970, 973, 1028-1049, 1090, 1117-1118, 1127-1136, 1226, 1241, 1244, 1249, 1268-1277, 1398-1399, 1443, 1452-1454, 1511, 1559-1569
plugins/search/cop_ghsl.py                     410      84  79.51%   73-74, 109-110, 174, 176, 218, 249-252, 298, 314, 360, 364-369, 384, 406, 427, 461, 477, 512, 527, 541-542, 556-558, 565, 569-571, 574-577, 595-655, 666-681
plugins/search/cop_marine.py                   297      65  78.11%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 294-297, 303, 313, 326, 334, 340-344, 350-353, 359-412, 428-431, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 246-287
plugins/search/eumetsat_ds.py                   25       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     889     108  87.85%   381, 425-426, 543-544, 567-568, 580-584, 816-822, 880, 946, 980, 987, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1214-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1435, 1450, 1523-1524, 1526-1528, 1530-1531, 1618, 1625, 1733-1737, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1933-1935, 1956-1962, 1969, 2007, 2023, 2046, 2051-2052, 2067, 2073, 2087, 2193, 2197, 2208, 2232, 2245, 2253-2263, 2289
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              637      58  90.89%   57, 186-241, 254-256, 274, 307, 387-391, 466-470, 550, 590-591, 620, 1002-1005, 1056, 1075-1076, 1108, 1126-1127, 1291, 1293, 1301, 1305, 1311, 1313, 1315, 1321, 1323, 1423, 1555, 1620, 1795
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             59       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                31       1  96.77%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    254      12  95.28%   200-203, 245, 263, 489, 536-537, 605, 680, 706
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11655    1392  88.06%

Diff against develop

Filename                                    Stmts    Miss  Cover
----------------------------------------  -------  ------  --------
api/product/_product.py                       +37       0  +0.80%
plugins/authentication/openid_connect.py       +8      +2  -0.55%
utils/exceptions.py                            +1       0  +100.00%
TOTAL                                         +46      +2  +0.03%

Results for commit: d4f4729

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Benchmark comparison

Compared baseline: origin/develop
Compared candidate: d17c306


------------------------------- benchmark: 7 tests, 2 sources -------------------------------
Name (time in ms)                                 baseline Median  candidate Median   ΔMedian
---------------------------------------------------------------------------------------------
test_benchmark_stac_reader_fetch_recursive                 3.5950            3.5760     -0.5%
test_benchmark_cli_version                                19.4400           19.5305     +0.5%
test_benchmark_cli_without_args_subprocess               131.4619          131.6437     +0.1%
test_benchmark_core_search_with_provider                 143.4981          143.1400     -0.2%
test_benchmark_core_search_with_count                    189.0283          186.8827     -1.1%
test_benchmark_cli_list_collections                      264.8707          270.8124     +2.2%
test_benchmark_eodag_instantiation_subprocess            961.4347          964.6912     +0.3%
---------------------------------------------------------------------------------------------

Legend:
  Cyan: reference source for comparison. Green: improvement, Red: regression.
  Δ: percentage change from reference source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants